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. Reflected diffusions in polyhedral domains are commonly used as approximate models for stochastic processing 

■ networks in heavy traffic. Stationary distributions of such models give useful information on the steady state 

performance of the corresponding stochastic networks and thus it is important to develop reliable and efficient 
algorithms for numerical computation of such distributions. In this work we propose and analyze a Monte- 
, Carlo scheme based on an Euler type discretization of the reflected stochastic differential equation using a single 

sequence of time discretization steps which decrease to zero as time approaches infinity. Appropriately weighted 
empirical measures constructed from the simulated discretized reflected diffusion are proposed as approximations 



for the invariant probability measure of the true diffusion model. Almost sure consistency results are established 



that in particular show that weighted averages of polynomially growing continuous functionals evaluated on the 
discretized simulated system converge a.s. to the corresponding integrals with respect to the invariant measure. 
Proofs rely on constructing suitable Lyapunov functions for tightness and uniform integrability and characterizing 
almost sure limit points through an extension of Echeverria's criteria for reflected diffusions. Regularity properties 
of the underlying Skorohod problems play a key role in the proofs. Rates of convergence for suitable families of test 
functions are also obtained. A key advantage of Monte-Carlo methods is the ease of implementation, particularly 
for high dimensional problems. A numerical example of a eight dimensional Skorohod problem is presented to 
C**) ' illustrate the applicability of the approach. 

oo ; 
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' 1. Introduction Reflected diffusion processes in polyhedral domains have been proposed as approx- 

imate models for critically loaded stochastic processing networks. Starting with the influential paper of 
Reiman[29|. there have been many works [28 ) H"2" ] 126 ] 136 ] 124 ] [35 ] that justify approximations via reflected 
diffusions rigorously by establishing a limit theorem under appropriate heavy traffic assumptions. Many 
performance measures for stochastic networks are formulated to capture the long term behavior of the 
system and a key object involved in the computation of such measures is the corresponding steady state 
distribution. Although classical heavy traffic limit theorems only justify approximations of the network 
behavior through the associated diffusion limit over any fixed finite time horizon, there are now several 
rcsults[THl [HI IH] that prove, for certain generalized Jackson network models, the convergence of steady 
state distributions of stochastic networks to those of the associated limit diffusions. Such limit theorems 
then lead to the important question: How does one compute the stationary distributions of reflected 
diffusions? fndeed, one of the main motivations for introducing diffusion approximations in the study 
of stochastic processing systems is the expectation that diffusion models are easier to analyze than their 
stochastic network counterparts. Classical results of Harrison and Williams [33] show that under certain 
geometric conditions on the underlying problem data, stationary densities of reflected Brownian motions 
have explicit product form expressions. However, once one moves away from this special family of models 
there are no explicit formulas and thus one needs to use numerical procedures. 

The objective of the current work is to propose and study the performance of one such numerical proce- 
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dure for computing stationary distributions of reflected diffusions in polyhedral domains. For diffusions in 
W n there are two basic approaches for computation of invariant distributions: PDE methods and Monte- 
Carlo methods. PDE approaches are based on the well known basic property that invariant densities of 
diffusions can be characterized as solutions of certain stationary Fokker-Planck equations. For reflected 
Brownian motions in polyhedral domains the papers|13 [ 1231 [TT] develop similar characterization results. 
The characterization in this case is formulated for the invariant density together with certain boundary 
densities and is given in terms of the second order differential operator describing the underlying un- 
constrained dynamics and a collection of first order operators corresponding to the boundary reflections. 
Using this characterization as a starting point Dai and Harrison[TT] develop an approximation scheme 
for the stationary density by constructing projections on to certain finite dimensional Hilbcrt spaces that 
are described in terms of the above collection of differential operators. Although PDE methods such as 
above are quite efficient for settings where the state dimension m is small, one finds that Monte-Carlo 
methods, based on the use of the ergodic theorem, have advantages in higher dimensions. With this in 
mind, we will propose and study here a Monte-Carlo method for the computation of stationary distribu- 
tions. Approximations of invariant distributions of diffusions in IR m using simulation of paths have been 
studied in several works [2] [27J [32j [3lJ [25] . One of the key difficulties in using simulation methods is that 
paths of diffusions cannot be simulated exactly and so one has to contend with two sources of errors: 
Discretization of the SDE and finite time empirical average approximation for the steady state behavior. 
In particular, the long term behavior of the discretized SDE could, in general, be quite different from that 
of the original system and thus a performance analysis of such Monte-Carlo schemes requires a careful 
understanding of the stability properties of the underlying systems. 

The Monte-Carlo approach studied in the current work is inspired by the papers [2], [27] . |25j which 
have analyzed the properties of weighted empirical measures constructed from a Euler scheme, based on a 
single sequence of time discretization steps decreasing to zero, for diffusions in IR m . For multi-dimensional 
diffusions with reflection one first needs to describe a suitable analog of an 'Euler discretization step'. In 
order to do so, we begin with a precise description of the stochastic dynamical system of interest. 

Let G C M. m be the convex polyhedral cone in M. m with the vertex at origin given as the intersection 
of half spaces i = 1, . . . , N. Let m be the unit vector associated with Gi via the relation 

d = {i£l m : (x,m) > 0}. 

Denote the boundary of a set S C !R m by dS. We will denote the set {x G dG : (x, rii) = 0} by F,. For 
x G dG, define the set, n(x), of unit inward normals to G at x by 

n(x) = {r : \r\ = 1, (r, x — y) < 0, Vy G G}. 

With each face Fi we associate a unit vector di such that (di,n,i) > 0. This vector defines the direction 
of constraint associated with the face Fj . For x G dG define 



Roughly speaking, the set d(x) represents the set of permissible directions of constraint available at a 
point x G dG. In a typical stochastic network setting this set valued function is determined from the 
routing structure of the network and governs the precise constraining mechanism that is used. This 
mechanism specifics how a RCLL trajectory tjj with values in K m is constrained to form a new trajectory 
with values in G, through the associated Skorohod problem, which is defined as follows. 

Let D([0,oo) : M. m ) denote the set of functions mapping [0, co) to M. m that are right continuous and 
have left limits. We endow D([0, oo) : K m ) with the usual Skorokhod topology. Let 



For r\ G D([0, oo) : M. m ) let |?7|(T) denote the total variation of r} on [0, T] with respect to the Euclidean 
norm on W n . 

Definition 1.1 Let G £> G ([0,oo) : M m ) be given. Then the pair (0,7?) G D([0,oo) : R m ) x D([0,oo) : 
R m ) solves the Skorokhod problem (SP) for -0 with respect to G and d if and only if <f>(0) = '0(0), and for 
all t € [0, oo) 




where 



In(a:) = {iG{l I 2,--.,JV}:(a:,n < >=0}. 



Az([0,oo) : R m ) = {0 G £>([0,oo) : R m ) : 0(0) G G}. 
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(i) <j){t)=i>{t)+r 1 {t); 

(ii) d>(t) e G; 
(in) \i]\(t) < oo; 

(iv) \n\(t) = J [Q t] I{4,( s ) ed G}d\7i\(s) ; 

(v) There exists Borel measurable 7 : [0, 00) — > R m such that ^(t) £ d(<p(t)), d\r)\-almost everywhere and 

n{t) = [ j(s)d\ V \(s). 
J[o,t] 

In the above definition <p represents the constrained version of tp and r\ describes the correction applied 
to ip in order to produce <j>. On the domain D C Z?g([0,oo) : R m ) on which there is a unique solutions 
to the Skorokhod problem we define the Skorokhod map (SM) T as T(tjj) = <fi, if (<f>, ip — tp) is the unique 
solution of the Skorokhod problem posed by tp. We will make the following assumption on the regularity 
of the Skorokhod map defined by the data {(di, n,); i = 1, 2, • • • , N}. 

Condition 1.1 The Skorokhod map is well defined on all of Dq([0, 00) : R m ), that is, D = Z?g([0,oo) : 
R m ) and the SM is Lipschitz continuous in the following sense. There exists a K < 00 such that for all 
^1,02 e D G ([0,oo):R™), 

sup |r(&)(f)-r(6i)(f)| <K sup - fo(t)\- 

0<t<oo 0<t<oo 

We will also make the following assumption on the problem data. 
Condition 1.2 For every x € dG, there is a n £ n(x) such that (d, n) > for all d £ d{x). 

The above condition is equivalent to the assumption that the N x TV matrix with (i,j) th entry (di, nj) is 
complete-S (see [TUESI])- When G = R™ and N = m, it is known that Condition 11.11 implies Condition 
11.21 (see [33]). An important consequence of Condition 1 1 . 2 1 that will be used in our work is the following 
result from [3] (see also [14] ). 

Lemma 1.1 Suppose that Condition \1.2\ holds. Then there exists a g € C^(R m ) such that 

(Vg(x),d t )>l \fx G Fi, i£{l,...,N}. (1) 

We remark here that the function constructed in [3] is defined only on G, however a minor modification 
of the construction there gives a C 2 extension to all of R m . 

We refer the reader to [101 [13 US] for sufficient conditions under which Condition 11.11 and Condition ll.2l 
hold. For example, the paper [TS] shows that if G = R™, N = m and the square matrix D = \d\, d m ] 
is of the form D = M{I — V), where M is a diagonal matrix with positive diagonal entries, V is off 
diagonal and the spectral radius of \V\ is less than 1, then both Conditions 11.11 and 11.21 hold. Here \V\ 
represents the matrix with entries (|Vy |), where Vij is the (i,j)-th entry of V. 

We now describe the constrained diffusion process that will be studied in this paper. Let (fi, F, P) 
be a complete probability space on which is given a filtration {Jjoo satisfying the usual hypotheses. 
Let (W(t),Ft) be a m-dimensional standard Wiener process on the above probability space. For x 6 G, 
denote by X x the unique solution to the following stochastic integral equation, 

X x (t) =T\x + J a(X x (s))dW(s) + J b(X x (s))ds^j (t), (2) 

where a : G — s- R mxm and b : G — >• R™ 1 are maps satisfying the following condition. 

Condition 1.3 There exists a\ € (0, 00) such that 

\a(x)-a(y)\ + \b(x)-b(y)\ <oi|x-y| Vx,y € G (3) 

and 

\<r(x)\<ai, \b(x)\<ai, Vx e G. (4) 
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Unique solvability of ^ can be shown using the above condition and the regularity assumption on the 
Skorokhod map. In fact, the classical method of Picard iteration gives the following: 

Theorem 1.1 For each x G G there exists a unique pair of continuous {J^t} adapted process 
(X x (t),k(t))t>o and a progressively measurable process (j(t))t>o such that the following hold: 

(i) X x (t) G G, for allt> 0, a.s. 

(li) For all t > 0, 

X x {t) = x + [ a(X x (s))dW(s)+ [ b(X x (s))ds + k(t), (5) 
Jo Jo 

a.s. 

(Hi) For all T G [0, oo), 

|/c|(T) < oo a.s. 

(iv) Almost surely, for every t > 0, 

= / I{xHs)£dG}d\k\{s), 
Jo 

k(t) = f*~f(s)d\k\{s), andj(s) G d(X x (s)) a.e, [d\k\]. 

In this work we are interested in the invariant distributions of the strong Markov process {AT 2 }. One of 
the basic results due to Harrison and Williams [2T] (see also [5]) on invariant distributions of such Markov 
processes says that if b and a are constants and a is invertible, then X x has a unique invariant probability 
measure if b G C° (the interior of C), where 

C = j-XJaidj : on > 0; i G {l,--- ,JV}j . 

This result was extended to a setting with state dependent coefficients in [T] as follows. We introduce 
the following two additional assumptions. For S G (0, oo), define 

C(5) = {veC :dist(v,dC)>6}. 



Condition 1.4 There exists a 6 G (0, oo) such that for all x G G, b(x) G C(S). 

Condition 1.5 There exists a G (0, oo) such that for all x G G and a G M. m , 

a {a{x)o~ {x))a > aa a. 

The following is the main result of [T|. 

Theorem 1.2 Assume that Conditions I 1111751 hold. Then the strong Markov process {X x (-);x G G} is 
positive recurrent and has a unique invariant probability measure. 

We remark that in pQ a somewhat weaker assumption than Condition 11.41 is used, which says that 
b(x) G C(5) for all x outside a bounded set. In the current work, for simplicity we will use the stronger 
form as in Condition 11.41 Conditions 11.1111.51 will be assumed to hold for the rest of this work and will 
not be explicitly noted in the statements of various results. 

We now summarize some of the notation that will be used in this work. For a Polish space S, V(S) 
will denote the space of probability measures, and A4f(S) the space of finite measures on S endowed 
with the usual topology of weak convergence. For a closed set G C K m , we say / G C£(G), [respectively 
/ G Cl(G)\ if / is defined on some open set O D G and / is a twice continuously differcntiablc on 
O with bounded first two derivatives [respectively compact support]. For v G V(S) and a i/-integrable 

/ : S — > M, we write J s fdv as (/, v) or v{f) interchangeably. We will use the symbol "=>" or "^>" 
to denote convergence in distribution. Let R m denote the set of m-dimensional real vectors. Euclidean 

P L p 

norm will be denoted by | • | and the corresponding inner product by (■, ■). The symbols, — >, — > denote 
convergence in probability and LP respectively. Denote by || • ||oo the supremum norm. A vector v G W 71 
is said to be nonnegative (and we write v > 0) if it is componentwise nonnegative. 
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1.1 Numerical Scheme and Main Results Throughout this work, the unique invariant measure 
for the Markov process {I 1 } will be denoted by v. The goal of this work is to develop a convergent 
numerical procedure for approximating v. We now describe this procedure. 

Let {Xk}k>i be a sequence of positive real numbers such that 

n 

X k — > 0, as k — > oo and letting A„ :~ A^, A„ — > oo as n — > oo. (6) 

fc=i 

Note the condition is satisfied if A„ = with 9 <G (0, 1]. Define the map S : G x M m — > G by the relation 

S(x,v)=T(x + vi)(l), (7) 

where i : [0, oo) — > [0, oo) is the identity map. The map S will be used to construct an Euler discretization 
of the stochastic dynamical system described by ([5]). We now introduce the noise sequence that will be 
used in the Euler discretization of ([5]). 

Let {Ukj; k G N,j = 1, ...,m\ be an array of mutually independent M valued random variables, given 
on some probability space J r , P), such that ~EU k j — and EL/| ■ = 1, for all k 6 N,j = 1, ...,m. Wc 
denote the M m valued random variable (Uk,i, Uk, m )' by We will make the following assumption 
on the array {U k j}. 

Condition 1.6 For some a £ (0, oo), 

Ee XUk -i < e aX2 for all k G N, j = 1, m, A G R. 

The above condition is clearly satisfied when Ukj ~ N(0, 1). Also, using well known concentration 
inequalities it can be checked that the condition also holds if supp(L r fe J ) is uniformly bounded (see 
Appendix for a proof of the latter statement). Condition 11.61 will be assumed to hold throughout this 
work. 

The Euler scheme is given as follows. Define iteratively, sequences {Xfc}fcgN j {^fe}feGN 01 G and M. m 
valued random variables, respectively, as follows. Fix xq G G. 

= xo, 

Y k+1 =X k + b(X k )X k+1 + a(X k )y/X^U k+1 , (8) 
_X k+ i = S{X k ,Y k+ i — Xk) ■ 

Note that {X k } is a sequence of G valued random variables. The last equation of the above display 
describes a projection for the Euler step that is consistent with the Skorohod problem associated with 
the problem data. 

Define a sequence of V(G) valued random variables as 

1 ™ 

v n = t— y~] Afe^Xit-i , n G N. 

k— 1 

The above random measures define our basic sequence of approximations for the invariant measure 
v. In particular, they yield an approximation for any integral of the form J G f(x)dv(x) through the 
corresponding weighted averages: 

1 " 

— £A fe /(AVi). (9) 

" fe=i 

The following is the first main result of this work. 

Theorem 1.3 As n — > oo, v n converges weakly to v, almost surely. 

The above result ensures that (O gives an almost surely consistent approximation for v{f) for any 
bounded and continuous /. In fact we have a substantially stronger statement as follows: 

Theorem 1.4 There exists a Q G (0,oo) such that for all continuous f : G — > M satisfying 
lim sup^^oo e - ^' |/(a?)| = 0, we have v n {f) — > v {f), a.s. 
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The key ingredient in the proof of the above almost sure limit theorems is a certain Lyapunov function 
that was introduced in [7] to study geometric ergodicity properties of reflected diffusions. Using this 
Lyapunov function we establish a.s. bounds on exponential moments of v n that are uniform in n. These 
bounds in particular guarantee tightness of {v n (uS),n > 1}, for a.e. tu. Then the remaining work, for 
proving the above theorems, lies in the characterization of the limit points of v n {uj). For this we use 
an extension of the well known Echeverria criterion for invariant distributions of Markov processes that 
has been developed in [13j [23] (see also [3] ) . Verification of this criteria (stated as Theorem 12.11 in the 
current work) for a typical limit point vq of {v n } requires showing that, vq along with a certain collection 
{//g,i = 1, • • ■ N} of finite measures supported on various parts of the boundary of G satisfy a relation 
of the form in (|27[) . The measures {fJ- l } are obtained by taking weak limits of certain finite measures 
constructed from the Euler scheme. Although these pre-limit measures may place positive mass away 
from the boundary of the domain, we argue using the regularity properties of the Skorohod map (a key 
ingredient here is Lemma ll.l[) . that in the limit these finite measures are supported on the correct parts 
of the boundary. 

Under additional assumptions, one can obtain rates of convergence as follows. For a > 0, set 

AW = A 1 +... + A«. 

Denote the normal distribution with mean a and variance b 2 by Af(a, b 2 ). For <fr G C 3 (G) (space of three 
times continuously differentiable functions on G) and v G W l , let D 3 (f>(x)(v)® 3 = ^ j t ^ij k < t ) i x ) v i v j v k- 

For / G C 2 (G), define Af : G R and A/ : G R; i = 1, ...,N as 

Af(x) = b(x) ■ V/(x) + ^a'{x)D 2 f{x)a(x), x e G, 

Dif(x) = d, ■ V/(x), x e G, 
where V is the gradient operator and D 2 is the m x m Hessian matrix. 



Theorem 1.5 Assume that Ui's are i.i.d with common distribution fi. There exists o ( 6 (0,oo) such 
that whenever 4> G C 2 (G) satisfies limi^i^.^ e - ^! |V0(x)| 2 = 0, we have the following: 



(a) Fast- decreasing step. Suppose \im r , 



A (3/2) 



0, D 2 (j> is bounded and Lipschitz, and 



Uv<f>{x),di) = 0, VxeF if Vi; 
yD 2 (j){x)d l = 0, VxeF^Vi. 



(10) 



Then the following CLT holds: 



k n v„{A4>) W 0, / ^V^dv 



(b) Slowly decreasing step. Suppose that lim„_ i . 00 (1/ V A„) A^ 3 ^ 2 ' = A G (0,+oo], <\> G C 3 (G) and D 3 (f> 
is bounded and Lipschitz. Further suppose that 



{V(j){x),di) = 0, VxeFi,\/i; 
D 2 (j)(x)di = 0, Vx€Fi,Vi; 
[D 3 jk 4>(x) ■ di = 0, Vie Fi,Vi,j, k. 



Then we have 



A 



k n v n {A<j)) ^ Af \^\m, j |cr J V&rdv 
™ -v n (A4>) ^> fn 



.3/2) 



if A < oo, 
if A = +oo, 



(11) 

(12) 
(13) 



where 



m 



D i (/)(x)(a(x)u) (Si fi(du)v(dx) 
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Note that when A& = A^^/v 7 ^ converges to [resp. oo, Ae (0,+oo)],ifa > 1/2 [rcsp. a < 1/2, 
a = 1/2]. Also note that if <f> is a smooth function supported in the interior of G then it automatically 
satisfies flTUJ} and (fTTjl. 

Proof of Theorem 11.51 is quite similar to that of Theorem 9 in |25j , the main difference is in the 
treatment of the reflection terms for which once more we appeal to regularity properties of the Skorohod 
map and an estimate based on Lemma 11.11 (see proof of (|33[) which is crucially used in proofs of Section 

A key step in the implementation of the Euler scheme in ([8]) is the evaluation of the one time step 
Skorohod map S{x, v). In Section 14.11 wc describe one possible approach to this evaluation that uses 
relationships between Skorohod problems and Linear Complementarity problcms(LCPs). There are many 
well developed numerical codes for solving LCPs (for example in MATLAB) and we will describe in Section 
14.21 some results from numerical experiments that use a quadratic programming algorithm for LCPs (cf. 
[TO] ) in implementing the scheme in ([8]). As remarked earlier, one of the advantages of Monte-Carlo 
methods is the ease of implementation, particularly for high dimensional problems. To illustrate this, in 
Section [4.21 we present numerical results for a eight dimensional Skorohod problem. 

The paper is organized as follows. In Section [2] wc prove Theorem II. 31 and II. 41 Thcorcm ll.3l is proved 
in two steps. Section [2~T1 shows the tightness of the random measures {f rl }, and Section [2~2l characterizes 
the limit of the measures {l^n} as the invariant measure of the constrained diffusion in @. Section |2~31 
gives the proof of Theorem 11.41 Rate of convergence theorem (Theorem ll.5[) is proved in Section [3] 
Finally we conclude by describing some numerical results in Section 2J 

2. Proofs of Theorems 11.31 and 11.41 The proof of Theorem II .31 proceeds by showing that for a.e. 
u>, the sequence of random probability measures {^ n (w)}n>i is tight and then characterizing the limit 
points of the sequence using a generalization of Echcverria's criteria. Tightness is argued in Section 
12.11 while the limit points are characterized in Section 12.21 Finally in Section 12.31 we give the proof of 
Theorem 11.41 

2.1 Tightness Wc begin by presenting a Lyapunov function introduced in [JJ that plays a key role 
in the stability analysis of constrained diffusion processes of the form studied here (see pQ [Tl [HI El El El H] ) - 

Throughout this work we will fix a 6 > as in Condition 11.41 

For x G G, let A{x) be the collection of all absolutely continuous functions z : [0,oo) — > M. m defined 
via 

z(t)=T [x + J v(s)dsj (i), i€[0,oo), (14) 

for some v : [0, oo) — »• C(6) which satisfies 

t 

\v(s)\ds < oo, for all t G [0, oo). (15) 

Define T : G — > [0, oo) by the relation 

T(x) = sup inf{i e [0, oo) : z(t) = 0}, x G G. (16) 

The function T has the following properties (see [JJ). 

Lemma 2.1 There exist constants c,C € (0, oo) such that the following hold: 
(i) For all x, y e G, 

\T(x)-T(y)\<C\x-y\. 

(ii) For all x G G, T(x) > c\x\. Thus, in particular, for all M € (0,oo) the set {x E G : T(x) < M} is 
compact. 

(Hi) Fix x € G and let z G A{x). Then for all t > 0, 



T(z(t)) < (T(x) - t) 
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We next present an elementary lemma that will be used in obtaining moment estimates. For k £ N, 
letF k = a(U u ...,U k ). Set T = {0, fi}. 

Lemma 2.2 There exist ci,C2 S (l,oo) for which the following holds. Let {fijigN be a sequence o/K. m 
valued random variables such that Vi is Fi—i measurable for all i > 1 and 

ess sup \vi(u)\ = \vi\oo < oo. 

w 

Let S n = X)™=i v i ' Ui, n€N. Then for every r > and n > 1, 

E max e r|Sl1 < c x e c ^ E "=i M« . 

l<i<« 

Proof. We will only give the proof for the case m = 1. The general case is treated similarly. 
From Doob's maximal inequalities for submartingales, we have 

E max e r|Sl1 < 4Ee r|s " 1 

l<i<n 

< 4(Ee rS " +Ee" rS ") 

From Condition 11.61 it follows that for every r£l, 

E(e rS "|J : - n _i) < e^-'e" 2 "" 

The result now follows by a successive conditioning argument. □ 
Define A : [0, oo) — » [0, oo) and j : [0, oo) — > No as 

A(s) = A fe ; j(s) =k, if A fe < s < A fe+1 , fc g N ; 
where we define Ao = 0. Define piecewise linear R m valued stochastic process as follows, 

t - A(i) 



^(*)= e v^c^ + - 7r =y=^(t )+ i, *>o. 

i<i(*) V a j(*)+i 
Let X{t) be the solution of the following integral equation 

X(t)=r(x + [ b(X{\{s)))ds+ [ a(X(\(s)))dW(s) J (f), t > 0. 



\ Jo Jo / 

Clearly, AT(A(f)) = X j(t) for all t > 0. 
Fix (0,1]. Define 

™ = * A = 4A + 16Lln( Cl ), (17) 

2(1 + p)L 

where i = C2a 2 G 2 i ; sr 2 and Ao = sup i>x A^. Let 1/ : G — > R+ be defined as 

V(o:) =e roT(3;) , ieG. (18) 

Lemma 2.3 There exist (3 € (0, 1) and </> S [0, oo) such that for each £ €E [0, p] and for all t > 0, 

E(U 1+ ^- (t+A) )| 7-- (t) ) < (1 - /3)F 1+c (X j(t) ) + (19) 

PROOF. Fix t > and ( e [0, p]. Define £ : [X(t), oo) -> G as 

■A(t)+- \ 



= r (x m + ^ 



6(X i(li) )du (s-A(i)), s>A(t). 

A(t) / 



Using the Lipschitz property of the Skorokhod map (Condition II. 1[) . we have 

a{X{\{u)))dW{u) 

X(t)+A+A J\(t) 

:KD(t, A). 



sup \X(s)-£(s)\<K sup 

A(t)<s<A(t)+A+A A(t)<s<A(i)+A+A 
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Note that 

A - A < X(t + A) - A(i) < A + A . 
Using this observation along with Lemma 12.11 (i) and (iii) , 

T(X(X(t + A))) <T(C(X(t + A))) + CKD(t, A) 

<(T(X(X(t))) - (X(t + A) - X{t)))+ + CKv{t, A) 
<(T(X(X(t))) - (A - A ))+ + CKD(t, A). 
From the above estimate and the definition of V(x), we now have 
V(V(X(X(t + A))y+<\F m ) 



< 



E (expMl + C) ((T(X(X(t))) - (A - A ))+ + CKu{t, A))) \T m ' 



V(X(X(tW+C 

xexp(~w(l + C))T(X(X(t))). (20) 



Letting, for q £ N , <r q = a{X q ), we have, for any s £ [X(t), X(t) + A + A ], 



EL S ,Y t ) Vqy/\+iU q +i, if a j{s )U j{s ) > 



a(X(X(u)))dW(u) < 
which can be bounded by 



£g=](t) Vq\/Xq + lUq + l, if (Tj( s )Uj( s ) < 



J 

9=3 (*) 



Similarly, 

r s 3 
- / a(X(A(u)))d#(u) < max - ^ <t ? VV^^+i- 



And therefore 



A) = sup 

A(t)<s<A(t)+A+A 



a(X{X{u)))dW(u) 
\(t) 



< max 

3'(*)<3<3t 



9=3'(*) 



where j t * = i(A(t) + A + A ). 

Using Lemma |2.2[ we now have that, with mo = w{\ + Q)CK, 

E [ e m ° p(t < A) | Jj (t) 1 < cie C2m » Q? E ?=^) A ' +1 < Cl e C2m ° a * ( A + 2A °). ( 2 1) 

In the case T(X(A(t))) > A - A , we have from J20|) and (J2TJ) that 

V(V(X(X(t + A))) 1+i \T j(t) ) < V(X(X(t))) 1+ < : e- ru(1+ ^ A - x ^ x Cie «T^^(A+2A ) < 
Recalling the choice of w and A, we now see that 

E(V(X(X(t + A))) 1+? |J r j(t) ) < (1 - /3)U(X(A(t))) 1+ <, 

where /3 = 1 — e _31nci . 

In the case T(X(X(t))) < A - Ao, we have 

E(^(X(A(f + A))) 1+C |j;- ( . f) ) < E (e moP( ^ A) |J- 3(t) ) < Cie c 2 m=a?(A+2A ) < ^^(A+SAo) = ^_ 

Combining the two cases, we have (fT9|) . □ 
The following lemma follows from Lemma 12.31 through a recursive argument. 

Lemma 2.4 There exists ai £ (0,oo) such that 

supE(U(A > (A(i))) 1+ ") < o 2 . (22) 
t 
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Proof. 

For any t £ (A, oo), we can find t' € (0, A] and j € N such that t = i! + jA. By a recursive argument 
using (fT9|) . we then have 

E(U(A(A(i))) 1+ ") < E(V(X(X(t')))^) + |. 

Thus 

supE(F(A(A(t))) 1+p ) < sup V{V{X{\{t))) 1+p ) 



0<t<A 



The supremum on the right side is bounded by maXj<j(A+A ) E(U(Aj) 1+p ), w hich is finite using Condition 
11.61 boundedness of b, a and the Lipschitz property of T. □ 

Now we can prove the following lemma. 

Lemma 2.5 For a.e. ui, sup n (V, v n {ui)) < oo. Consequently, the sequence {v n {u)} n >i is tight for a.e. u. 

Proof. Let no be such that A no > A. Then it suffices to consider the supremum in the above display 
over all n > uq. For i <G No, define s(i) = inf{j E No : Aj > iA}. Then s([A n /AJ) < n and therefore, for 
n > no, 

u n(v) =j- ]T ^y(Xk-i) = j- / v(x(\(t)))dt 



1 



(|A„/AJ+1)A 



<- / V(X(X(t)))dt. 

A s(|A„/AJ) Jo 



Using Lemma \2 . 31 with £ = 0, we have 
1 



A S (|A„/AJ) Jo 



(LA„/AJ+1)A 

V(X(X(t)))dt 



1 

< 



A S ([A„/AJ) Jo 



1 



A S (|A„/AJ) Jo 



(LA„/AJ + 1)A 

[V(X(X(t))) - E(V(X(X(t + A)))\T m )]dt 

(|A«/AJ+1)A 

[V{X{\{t))){l-p) + <l>]dt. 



Thus, rearranging terms, 

(3 



A S ([A„/AJ) Jo 



(LA„/AJ+1)A 

V(X(X(t)))dt 



1 

< 



A s (|_A n /AJ) Jo 



(LA„/AJ+1)A 

[V(X(X(t))) - E(V(X(X(t + A)))\T m )}dt 



(LA W /AJ+1)A 
+ <f>— : ■ 

A s(L A„/Aj) 

Next note that A s (ia/a|) > [A^/ AJ A, and, for n > n , 

A s ([a„/aj) > A s (i) > Ai. 

Thus 

, (|A„/AJ+1)A A, 
sup </> VL n/ J '— < <j){l + — ) < oo. 

n>n A S (LA„/AJ) A l 

To prove the lemma, it is now enough to show 

»(LA„/Aj+1)A 



sup- / ' [V(X(X(t)))-^(V(X(X(t + A)))\T m )]dt <oo, a.e. a;. (23) 

n>n A s(LA„/AJ) Jo 
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The above expression can be split into two terms: 



1 



A S ( L A„/Aj) JO 



(LA„/Aj + 1)A 

[V(X(X(t))) - B(V(X(X(t + A)))\T m )}dt 



1 



A S (|A„/AJ) Jo 



(|A n /Aj+l)A 

[V{X(X(t))) - V{X(X(t + A)))]dt 

(LA„/AJ+1)A 

[V(X(X(t + A))) - E(V(X(X(t + A)))\T m )]dt 

A S (LA„/AJ) Jo 

= Ti + T 2 

Consider the first term: 

P (LAn/AJ+l)A i /■(|A n /AJ + l)A 

T x =- : / V(X(X(t)))dt-- / V(X(X(t + A)))dt 

A S (|_A„/AJ) JO 

(LA„/Aj+1)A i ,(LA„/Aj+1)A+A 

V(X(X(t)))dt - / V(X(X(t)))dt 

A s(LA„/AJ) J A 
A i ,.(|A„/AJ+1)A+A 

V(X(X(t)))dt-- / V(X(X(t)))dt 

A S (|A„/AJ) J(|A n /AJ + l)A 

<t " E 

A S ( L A„/Aj) , :A ^< A 

Let Z = S fe . Afc _ 1<A A fc V A (A' fe _ 1 ). Then from (gl]), we have EZ < a 2 (A + A ). Combining this with 
the fact that for n > n , A s (ia /aj) > -Vt, we have that 

sup Ti(w) < oo, a.e. ui. (24) 

n>no 



f 


A ^(LA„ 


/AJ) 


1 




A .s(LA„ 


/AJ) 


1 




A .s(LA„ 


/AJ) 


f 




A s(LA„ 


/AJ) 



Next, consider T 2 : 



f 



A s(LA„/Aj) Jo 



(LA„/AJ+1)A 

[V(X(X(t + A))) - E(V(X(X(t + A)))\T m )}dt 



A s(LA„/AJ) -~Q JiA 



LA„/AJ ( i+1 ) A 

E / [n^(A(i + A)))-E(y(l(A(t + A)))|J- J . (t) )]^. 

„•_ n JiA 



From Kronccker's Lemma (see page 63 of |17j). the last sum is bounded in n a.s. (in fact converges to 0) 
if the following series is summable a.s. 

Er- / [V(X(X(t + A)))-E(V(X(X(t + A)))\T m )]dt. 

7~ x A s(i) JiA 



Consider the sum over even and odd terms separately. For even terms, the sum can be written as 

00 -I A2k+1)A 

Ea / [V(X(X(t + A)))-E(V(X(X(t + A)))\T m )]dt. (25) 

r n A s(2k) J2kA 



fe=1 ^s(2k) J2kA 

Let 

/.(2fe+l)A 

&+1 = t / [V(X(X(t + A))) - V(V(X(X(t + A)))\T m )]dt 

A s(2fc) J2kA 

and Qk = J 7 j( 2 kA), then we have E(^+i|5i) = 0. Also note that is Gi+i measurable. Thus S n = 
S"=i & i s a martingale with respect to the filtration {Gn}- Consequently, by Chow's Theorem (see 
Theorem 2.17 of [E]), the series in (|25]l is a.s. summable if Sfcli E(|£fc| 1+P ) < oo. Now note that 



E|a| 1+ " =E 



A s(2fe) J2fcA 



(2fc+l)A 

[V(X(X(t + A))) - E(V(X(A(i + A)))|^- (t) )]dt 



1+pN 



7V s(2fe) 1 iV s(2fc) 
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where the last inequality follows form Lemma 12.41 Since A 5 (£) ^ we have that 

1 11 

fc=l fc=l 

This proves that the series in (|25p is summable. The odd terms are treated in a similar manner. Thus 
we have proved 

sup T 2 (lo) < 00, a.e. w. (26) 

ri>n.Q 

Now (f23|) is an immediate consequence of ((24)) and (|26|) . which proves the lemma. □ 

2.2 Identification of the limit In this section we will complete the proof of Theorem ll.3l bv arguing 
that for a.e. w, every weak limit point of v n (ui) equals v. For this we will use the following extension of 
the Echeverria Criteria (see [23l|34], see also Theorem 5.7 of [3]). 

Theorem 2.1 Let v g V(G) and g M.F(Fi), i = 1,.~,N be such that for all f g C%(G), 

N 

MAf) + Y,t4>( D if) = °- ( 27 ) 

i=l 

Then i/q = v. 

In order to apply the above theorem to show convergence of v n to v : we will consider a sequence of 
finite measure {/i^JneN! i = 1, N, which, roughly speaking, correspond to the prelimit versions of the 
measures {/Iq} that appear in the theorem above. We now describe this sequence. 

For u £ R m , v g G, re (0,oo), define, for f g [0, 1], 

z(«,v,r|i) = z(f) = w + (b(v)r + a{v)y/ru)t, 
x(u,«,r|t) = x(t)=r(z)(t), 
y(u,u,r|t) = y(t) = x(t) - z(t). 

Then, one can represent the trajectory y as 



y(*) = f> f ai(s)d\y\(s); te [0,1], 



(28) 



where a,(s) = aj(u, u, r\s) g [0, 1] and «j(s) > only if x(s) g .F,. Also, let, for t g [0, 1] 

n t ( U ,«,r) = z(l) + i(x(l)-z(l)), 

L i (u,«,r)= / Oi^dlyKt), * = 1,...,JV. 
Jo 

Finally for k g N , let 

life = ^(f/fe+ijX/s, Afc+i), ife = L^C/fc+ijXfc, Afc+i). 
For k g No and i = 1, AT, define a .M F(R m ) valued random variable ml by the relation 

ty,mi) = [ ExMOJtilfodt, ^eBM + (n (29) 
Jo 

where Ex [2] denotes E[Z|X], and BM + (W n ) is the space of nonnegative bounded measurable functions 
on R m . 



For n g N and i = 1, JV, let y^ be & Ai f(l m ) valued random variable defined as 

n-l 



1 n — 1 

(A) = —J2<(A); A g S(M m ). 



A„ 

" fe=0 



The following lemma relates the above family of random measures with our approximation scheme. 
Recall the definition of the filtration {Ft} in Section |2~T1 
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Lemma 2.6 For every f G C^(W n ), there exists a sequence of real random variables {Cnl^eN such that 
n-i n 
— J2 E[/(X fe+1 ) - f(X k )\F k ] = Y,&(Pif) + u n (Af) + (30) 

™ fe=0 i=l 

and sup„ < oo a.s. Furthermore if f has compact support then — > a.s. as n — > oo. 



Proof. Fix (u, v, r) <G R m x G x (0, oo). Using the notation introduced above, we have from Taylor's 
theorem, 

/(z(l)) - f(v) = (V/(«), 77) + \v'D 2 f{v) V + R 2 (v, z(l)) 

where 

R 2 (x, y) = f(y) - f(x) - (V/(x), y -x)-±{y- x) T D 2 f(x)(y - x) 

and 

r) = i](u, v, r) = b(v)r + a(v)y/ru. 

Define 

r 2 (x,y) = l sup \\D 2 f(x + t(y-x))-D 2 f(x)\\, 
1 te(o,i) 

then we have |i?2(^, 2/)| < ^(^j y)|x — y| 2 . 
Also 

m^-m^^j ^^^-^ oit 

1 V/(z(l) + t(x(l) - z(l))) • (x(l) - z(l))dt 

= V/ V/(z(l) + i(x(l)-z(l)))di-di / o<(t)d|y|(t). 
i=1 Jo Jo 

Fix a /€ G N and let v — X k , u = Uk+i and r = Afc+i. Then 

E[/(X fc+1 ) - f{X k )\F k ] = E[/(x(l)) - /(z(l)) + /(z(l)) - /(«)|^ fc ]. 

From the definition of mi in (|^| and observing that {X k } is a Markov chain (with respect to the 
filtration {Fk}) and U k +i is independent of F k , it follows that 

N 

E[/(x(l))-/(z(l))|J- fc ]=^ml(A/), 

t=i 

Using independence of C/jt+i from J-^ once more, 

E[/(z(l)) - }{v)\F k ] =X k+1 (Vf(X k ),b(X k )) + ±X k+1 a(X k )'D 2 f(X k )<j(X k ) 

1 

+ 2 

=A fe+1 A/W+£ / W 

where 



-X 2 +1 b(X k )'D 2 f(X k )b(X k ) + E[R 2 (X k ,X k + n k )\F k ] 



^(fc) - \\ 2 k+1 b(X k )'D 2 f{X k )b{X k ) + E[R 2 (X k ,X k +r] k )\T k ] 



and v k = r](U k+1 ,X k ,X k+ i). 
Thus we have 



. n— 1 

-^E[/(X fc+1 )-/(X fe )|J- fe ] 



1 n— 1 



A' 



]T r4(A/) + A fc+ i^/(X fc ) + 

n 1 n— 1 

J2l4 l (D i f) + u n (Af) + — ^2e(k). 



k=0 
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Equality in flU]) follows on taking ^ = -±- J^lZl £ f {k). 
We now show that sup n £^(o;) < oo a.s. Write 

t f (k) = ^Xt +1 b(X k yD 2 f(X k )b(X k ) + -E[R2(X k: X k + m )\T k } =^{(k) + ^(k). 

The term X)fc=o £1 W converges to zero because of the boundedness of b and D 2 f. Consider now the 
contribution from (k). Let for p £ W.+ , 

1 



\Xl— X 2 \<P 



h(p) = - sup \\D 2 f{x 2 )-D 2 f{ Xl ) 

^ x 1 ,x 2 eS. m 



Then 

\R 2 (X k ,X k + r ]k )\ < h( Vk )\ Vk \ 2 

and so for some Ki £ (0,oo), 

|^(fc)| <nh(vk)\m\ 2 \^k] 

Thus sup„ Cn( w ) < 00 a - s - This completes the first part of the lemma. 

Finally if / in addition has compact support, we have h(p) — > as p — > 0. Fix e > 0. Since b, a are 
bounded, we can find for each 6 £ (0, 00), kg £ N such that for every k > kg, 



\h{b{x k )X k+1 +a{xk)\/ Afe + i U k+1 ) \l\ Uk+1 \<e < e - 
Also, for some l v £ (0,co), for all k £ N, 

E[\ij k \ 2 \T k ] <l n X k a.s. 
Choose do € (0,oo) such that E[| L^i | 2 1| ^ |> 0O ] < e. Then 

n—l -. n—l -. n—l n—l 

T E \^)\<d vT - E A* + INW,(yT E A fe /2 +A - E A ^)- 

k=ke () k=kg Q k=kg Q k=kg Q 

Thus, 

1 n-1 fe e _1 -, n-1 

x;Ei^wi<x; E ^(*) + ^(i + ni=o) + ini=oI^ E A '- /2 - 

fc=0 A'=0 k—kg^ 

Sending n —> 00 and then e — > 0, we now see that Y^k=o 1^1(^)1 — > as n — > 00. The result follows. 
□ 

The following lemma shows that the left side of the expression in (f30f converges to as n — > 00. 
Lemma 2.7 For every / e C 2 (G), 

n-1 



— E E[/(X fc+ i) - /(X fc )|J" fe ] -> a.s., as n^oo. 



Proof. Wc can split the sum into two terms: 

n—l 

— EE[/(^+i)-/(X fc )|J- fc ] 



A » fe= 



n—l n—l 

=— e (E[/(**+i)i ^] - /(jffc+i)) + t- E (/(**+o - /(**)) 



" fc=0 n fc=0 

=T!+T 2 . 
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Note that, 

\T 2 \ = ^-\f(X n )-f(X )\^0, 
as n — > oo, since / is bounded and A„ — > oo. Also, using Kronecker's Lemma, 

n-l 



T i = T~ E (E[/(* fc+ i)l^] - f(Xk+i)) 
A " fc=0 

will converge to once the martingale 

n— 1 

M n ■= E T-(E[/(X fe+1 )|J" fe ] - f(X k+1 )) 



converges a.s. Finally observing that E(/(A"fe_|_i)|J r fc) minimizes the L 2 distance from f(Xk+i) among Fk 
measurable square integrable random variables, 



E<M') 00 = E(]f) 2E (f( X ^) E(/(X fc+1 )|J- fe )) 2 
fc>i fc 

<]T(^) 2 E(/(X fc+1 )-/(X fc )) 2 
fc>i k 

<iia/iuE(a L ) 2e ^'+ i - x ^ 2 



fc>l K 



fc>l fe 



< oo 



for some constant Hi , where the last inequality follows from the observation that for a positive sequence 
^k, J2k>i ^fc+i/A 2 , < oo. The lemma follows. □ 

Next we consider the limit of the first term on the right side of ([50)1 . We can regard \i % n to be a finite 
measure on the one point compactificaion of R m , denoted as R m . In order to show that {/i^} is a.s. a 
precompact sequence in A4_f(R" 1 ), it suffices to show that /i^(R m ) is an a.s. bounded sequence of Re- 
valued random variables. This is shown in the following lemma. 

Lemma 2.8 For i = 1, N, 

sup/4(R m ) < oo, a.s. 

n 

PROOF. Let g g Cf(R m ) be as in Lemma ITTT1 Then for fixed (u,v,r) <G R m x G x (0, oo) and with 
notation as introduced above Lemma 12.61 



<?(x(l)) =g(v) + / [Wg^(s))-(b(v)r + a(v)^u)}ds + y2 / <U ■ V 5 (x(s))a i ( s )d|y|(s) (31) 
Since ai(s) is nonzero only when x(s) € i^, and (\7g(x), di) > 1, for all i 6 fj, i 6 {1, Af}, we have 



£l/( W )=£ / a !; ( S )d|y|( S ) 
i=i i=i - 70 

5^y o di-Vfl(x(*))ai(*)d|y|(«) (32) 



AT 

=1 

< | 5 (x(l)) - g(v)\ + ||V 5 || 00 |6(7 J )r + a(v)y/hi\ 

< ||Vfl||oo|x(l) ~v\ + \\yg\\oo\b(v)r + a(v)y/fu\ 

< \\Vg\\oo(K + l)\b(v)r + a(v)^u\, 

where the second inequality uses (|31[) , and the last inequality uses the Lipschitz property of the Skorokhod 
map. 
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Let ki = 1 1 V5I |oo(iir + l)oi, then from (|32J| we have for i G {1, ...,N], 

L{ < ki (v^+T|l/fc+ll + A fe+i) • (33) 

Also note that, 

sup \x k (t) -X k \< K\b(X k )X k+1 + a(X k )^/X k ~^U k+1 \ < K ai ^/X^\U k+x \ + K ai X k+u (34) 
te[o,i] 

and for t G [0, 1], 



K~X k \ <t\x k {l)-v\ + (l-t)\z k (l)-v\ < (K + l) ai \ k+1 + (K + l)a iy /j^\U k+1 \, (35) 

where x k (t) = n(U k+ i,X k , X k+ i\t), z k (t) = z(U k+ i,X k , X k+ i\t). Combining (|3"3" ]) -(|3"5" ]l we have that 

E Xfe (|n* -x k (s{)\Li) < (2K+l)a lKl mX k+1 + V (X k+1 )X k+1 , (36) 

where tp : (0, 00) — > (0, 00) is a bounded function satisfying <p(a) — > as a — > 0. 

Next note that L\ is not equal to only if there exists s G [0, 1] such that a k (s) > 0, i.e., x k (s) G F,- L , 
where a k (t) = ai(U k +i, X kl X k+ \ \t). And in that case, 

D. ig (Ul) > D t g(x k (s)) ||^ 2 ff ||oo|n* - x k (s)\ > 1 - ||D 2 <?|Un< - x k (s)\. 

Let A\ = {uj : there exists s G [0, 1] such that a k (s) > 0} and 



4M = 

Then, from (|55]). 



inf{.s G [0, 1] : a k (s) > 0} if u G A k , 
1 X«jtA\. 



E Xh [D i9 (Ul)Lil Al } > E Xk [L\l Ai ] - WD^lUExM ~ **(4)l4l 



At 

2 . 



> E Xfc [LI] - || J D 2 3 || 00 ((2X + l)a lKl mX k+1 + <p(\ k+1 )X k+ i) 
Thus we have 



(D ig ,mi) = [ ExjA5(n* fe )4]^ 
Jo 

= jf E x JAs(nt)4iAi]^ 



> (l,m|) - ||£> 2 g||oo ((2if + l)aimKiA fe+ i +¥>(A fe+ i)A fc+ i) . 
Rearranging the terms, we have 

(l,m|) < (D ig , m k ) + \\D 2 g\\ oa ((2K + l)a imKl X k+1 + ip(X k+1 )X k+1 ) . 
Summing over k from to n — 1 and i from 1 to iV, we obtain 

N N 

A*j,> < 52{Di9, /O + VV||^ 2 .9lloo ((2K + l)oi/c im + Moo) . (38) 

»=I i=l 

Using Lemma |2. 61 

N n-1 

E ^(Afl) = t- E E fo(**+i) - fl(^fc)l^fc] - "n(Ag) - ^ 

i=l n k=0 

Since g G C£(M. m ), the second term on the right side is bounded. Also from Lemma [2.71 the first term 
converges to as n — > 00. Finally from Lemma 12.61 the third term is bounded, a.s. 

From this it follows that 

N 

sup /j, l n (Djg) < 00 a.s. 

n 

Result follows on using this observation in ([55)1 . □ 

The following lemma will be used to show that for a.e. w, any limit point of fi l n {u>) is supported on Fj, 
t = 1,...,JV. 
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Lemma 2.9 Fix i e {1, ...,N}. Let ip e C^{R m ) be such that ip(x) > for all x G R m . Suppose that 
there is a e > 0, such that ip(x) = if dist(x, F,) < e. Then 

ip{x)^ 1 n (dx) — > 0, a.s. as n — > oo. 

Proof. We have 

_^ ?i— i 



A„ 

fc=0 
n— X 

a" 



/ E A - fc [^(n' fe )r fc ]* 

^ fc=o - 70 

-J2 / E A - fc / ^(n*,)af( S )d|y fc Ut 



(39) 



A„ 

fe=0 



l"0l ™ _1 f 1 



o Jo 



where, recall that a k (s) = ai(C/ fc+ i, X k , X k +i \s) and x fc (s) = x(U k+1 ,X k ,\ k +i\s), y k (s) = 
y(U k+ i,X k , X k+ i\s). The last inequality in the above display follows from noting that a k (s) > only 
when x k (s) £ Fi and if for such a s, \U k — x k (t)\ < e,we have by our choice of ip that 4>(n. k ) = 0. 

Next note that 

{(t,s,u) : \Ul - x k (s)\ >e}c {(t,s,uj) : \x k (l) - x k (s)\ >e}U {(t,s,uj) : \z k (l) - x k (s)\ > e}, 
where recall that z k (t) = z(U k+ i, X kl \ k+ i \t). 

Also, from the Lipschitz property of the Skorokhod map, 



\x k {\) - x k (s)\ < Ka x \ k+1 + Kax^Xk^lUk+il 

and 

|z fc (l) - x k {s)\ < \z k (l) -X k \ + \X k - x k (s)\ < (K + l)oiAfc+i + [K + lJoiv/A^iTl^iH-il- 

Thus 

{u : |nt - x k (s)\ > e for some t, s E [0, 1]} C {uj : \U k+1 {w)\ > p k }, 
where p k = £ /((- R: +|) ai ) At +! _ Using this observation in (|39|) , we have 

(V,^)<^E E ^ f f h\u\> Pk} a k {s)d\y k \ s dt 
An k=0 
n-l 



Jo 



1V n fc _ n Jo 



A 



fe=0 
n-l 

E 



n fc=o\ 



Ex fc a k (s)d\y k \^J )p(\U\>p k ) 



From (|3"2"T) it follows that for some Ki € (0, oo), sup fe E At (^Jq a k (s)d\y k \g\ < Ki. Also using Condition 
1 .61 E|Z7p < oo for all j > 1. Choose fc large enough so that Afc+i < 2 (K+i) ai ^ or au ^ — ^*°- 
Fix j > 4, then 

(v , ) /4) < ^v**> + ^ £ 0^(E|^py/v 72 - 

k—KQ 

The result now follows on observing that for some Ki £ (0, oo), pl o/2 < K 2 A^ for all ft > fc . □ 
We are now ready to complete the proof of Theorem 11.31 
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Proof. [Proof of Theorem 11.3] Fix / £ C^(G). Then such a function can be extended to a function 
in (K m ). We denote this function once more by /. Then from Lemma [2.61 



j ra-l N 

—j2nf(x k+ i)-f(x k )\^ k } = j2^(Dif)+M^f)+^- (40) 

n k=0 i=l 

From Lemmas 12.51 12.61 12.71 and 12.81 there exists fto £ J- such that P(flo) = 1 and for every u> £ f2o, 

• {v n (u})}n is precompact in V(G), 

• {fj, l n (uj)} n is precompact in _A/[_p(]R m ), for every i — 1, ...,N, 

• Left hand side of (|4"0]l converges to 0, 

• ££(w) converges to 0. 

Fix a to £ Qq and let ^(w), /^(w), i = 1, iV, be a subsequential limit of ^„(u;) and ^ n (uj), respectively. 
Then from ()40p and the above observations, we have ( suppressing uj ) 

N 



^W) + ^^(A/) = o. 



To complete the proof, in view of Theorem 12. 1\ it suffices to argue that 

l F c(x)iii o (u)(dx)=0. (41) 
By convergence of fi l n to /Lt^ , we have for every tp as in Lemma 12.9) 



Therefore 

/ l^.r(x)ni B (dx) = Ve,r>0, 

where F/' r = {a; £ K m | dist(x,i^) > e and \x\ < r}. The equality in (|4T|) now follows on sending e — > 
and r — > oo. □ 

2.3 Proof of Theorem [01 Recall c from Lemma O and tu from (fl7|l. Fix C £ (0, roc). We will 
prove the theorem with such a choice of f. Consider an / as in the statement of the theorem. Then there 
exists constant K\ such that < Kie^> x > Without loss of generality, we assume / > 0. 

From Theorem ll.3[ f° r an Y £ > 0, we have 



(/ A L)dv n -> / (/ A £)df a.s. 
In order to prove the theorem, it suffices to show that 

J (f A L)di> n — > J fdu n , and /"(/ A L)dy — > / /di/, as L — > oo. 

First, consider 



sup 



/ fdu n - J {f A L)d 



< sup / \f > h]dv n 

n J 

<su P > i)K(/ 9 )] 1/9 ) , 



where p,q £ (1, oo) are such that p 1 + g 1 = 1 and the last inequality follows from Holder's inequality. 
Choose q > 1 such that £q < ccc, then from Lemma 12.51 we have 



svLp[v n {f q )} 1/q < kisu P [ / e^ x ^ n (dx)] 1/q < KiBapuV*(V) < oo, a.s. (42) 

n n J n 

Using Markov's Inequality, we have 

> L) < ^f-, 
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which using (|42[) converges to as L goes to infinity. Combining the above three displays, we have 



sup 



fdv n - I (/ A L)dv n 



— > 0, a.s. as L — s- oo. 



Also, from Fatou's lemma we have, for a.c. uj, 

[ fdv - J{f A L)dv = J (/ — / A L)dv 

< liminf J (/ — / A L)di/„ 

< sup / (/ — / A L)d^„. 

n J 

Using (|43|) the last expression converges to as L — )• oo. The result follows. 
3. Proof of Theorem 11.51 We begin with a few preliminary lemmas. 

Lemma 3.1 If 4> e C 2 (G), then 

n 4 4 



k=l 



with 



Z<°> = ftXj - <f>(X ), 
n 



fc=i 



i ™ 

- J] A2&(*k-i) r DV(**-i)&(*Jb-i). 



fc=i 



^Af^n) T B^ft-i)a(I S -i)(/ t 



fc=i 



n 

4 3) = -^A fe [( ( 7(X fc _ 1 )(7 fe ) T J D 2 ( /)(X fe „ 1 )( ( T(X fc _ 1 )[/ fe 



fc=i 



-E((a(X fc „ 1 ) l 7 fc ) T Z? 2 ( /»(X fc _ 1 )(a(A' fe _ 1 )[/ fe )|J- fc _ 1 )], 



fe=i 



and 



(43) 



- £(v<havi),^-i), 
fc=i 

n 

fc=l 

n 

r i 3) = 5Z Afc6 (- y *- i ) Tx>2 ^ jr *- i )f*- 1 ' 

fe=i 

n 

fc=i 

w/iere i? 2 (x, y) = 0(y) - 0(a:) - (V0(x),y- x) - | (j/ - x) T D 2 cj>(x)(y - x), and y k = y(U k+ i,X k , Afc+i|l). 
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Proof. Denote S<f>(X k ) = 4>(X k ) - <j>(X k -x) and 6X k =X k - X k _ X - We deduce from © that 
84>{X k ) =(V^»(X fe _ 1 ),5X fe ) + ^6X k r D 2 <f>(X k _ 1 )6X k + R 2 (X k _ 1 ,X k ) 



=(V^_i),!/n)+A t ^(In) + VA t (V^(In),(r(I M )[/ fe ) 
+ \y T k ^D 2 ^X k ^)y k ^ + iA 2 fe 6(X fe _ 1 ) T ^ 2 0(X fe _ 1 )6(X fc _ 1 ) 

+ ^\ k [{a{X k „ 1 )U k ) T D 2 4>{X k „ 1 ){a{X k „ 1 )U k ) - ^{{a{X k ^)U k ) T D 2 cp{X k ^){a{X k ^)U k )\ F k ^)] 

+ X k b(X k ^f 'D 2 cf>(X k _ 1 )y k _ 1 + \l /2 b(X k ^ 1 ) T D 2 (b(X k _ 1 )a(X k ^ 1 )U k 
+ y/\ k 'y^_ 1 D 2 4,(X k . 1 )a(X k _ 1 )U k + R2(X k - 1 ,X k ). 
The lemma follows by summing the above equality over k = 1, n and rearranging the terms. □ 

Lemma 3.2 Let W : G — > R be a continuous function such that sup IlSN v n (W) < oo, a.s. Let <fi € C 1 (G), 
be such that lim|, I ,|__>. 00 |V0(a;)| 2 /W / (a;) = 0. Then 



A » fc =i V Jg J 



Proof. This lemma follows from Theorem II .31 using the martingale central limit theorem, along the 
lines of Proposition 2 of [2S]. Details are left to the reader. □ 

Lemma 3.3 Under the assumptions of Theorem \1.5\f b), we have, 

7(4) o 1 /* /" 
"TO r / D^(x)(<j(x)ur^(du)v(dx) 7 
A£ /2 > 6 J G J Rm 

as n — > 00. 

Proof. The proof is similar to that of Lemma 10 of {25] except for the treatment of reflection terms. 
Using the notation above Theorem 11.51 and in Lemma l3.1[ we have 

R 2 (x, y) = l -D^{x)(y - xf 3 + R 4 (x, y), (44) 
6 

with 

\Ri(x,y)\ < ^\y-x\\ 



where L is the Lipschitz constant for D 3 <f>. Hence 

R 2 (X k _ 1 ,X k ) = iD 3 0(Xfc_i)(*X fc )® 3 +r fc , (45) 
6 

with 

\r k \ < ^\SX k \ 4 < Kl X 2 (l + |C/ fc | 4 ), k e N, 
for some Ki <E (0,oo). Since E|£/ fc | 4 := ^ 4 < 00 from Condition 11.61 we have 

n n 

E^|r fc |< Kl (l + M4 )^A 2 fc . 

k=l k=l 

From the assumption lim„^ 00 (l/v / A^") J2 k =i ^ k = ^ G (0, +00], we deduce that limn^oo Y^k=i ^k" 2 = 
+00 and 



hm ^A 2 /A( 3 / 2 )=0. (46) 

n — inn — * 

Therefore, 



n— yoo ' 



1 71 



A(3/ 

Ji ™ fc=l 
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Now consider the first term on the right side of f|45j ). 

£> 3 0(X fc _i)(£Y fc )® 3 =D 3 cj>{X k _ l ){X k b{X k _ l ) + ^X~ k a{X k ^)U k + y k ^) m 

=X 3 k /2 D 3 cf>(X k _ 1 )(s/X k ~b(X k _ 1 ) + cj{X k ^)U k f 3 + fi 1} (Xk-i, U k ) 
=X 3 k /2 D 3 4,(X k _ 1 )(a(X k _ 1 )U k f 3 + f { k 2) {X k _ u U k ) + f^iXk-uUk), 

where f k (X k _i, U k ) and fj^\X k -i, U k ) are defined through the second and third equalities, respectively. 
Next observe that 

• From the assumptions, we have 

\X k b{X k ^) + v^trOYfc_i)E/-fc| < axVAldLTfel + y/\o). 

• From ((25)) and we have y k -\ = Y^iLi diL k _ 1 and for some k 2 £ (0, oo), 

4-1 < «2V^(l^fc| + 1). for all e N 

• The term L l k _ 1 is non zero only if there exists s 6 [0, 1] such that x k -i(s) G Fj, where x k -i(s) = 
x(U k ,X k -i,X k \s). And in that case, we have from the Lipschitz property of D 3 (f> and (fTTj) 
that, for some k 3 € (0, oo), 

IDl^pk-O • d*| < « 3 >/Afc(|Cfcl + 1), Vj,fc. 

Combining these estimates, we see that E X]fc=i l/fe (-^fc-ii ^fc)| < K 4 X)fe=i ^fc- Using (j46| we now have 

-pi^/PtlH.t/^O. (48) 
A " fe=i 

For the term f [ k ] {X k _ u U k ), using the boundedness of D 3 cf), 6, and a, it can be easily checked that 
nf ( k \x k _ u U k )\ < K 5 X 2 k . Thus 

n n 

Ej2\MXk-i,U k )\ <k b J>|, 
fe=i fc=i 

and so using (|4l)|) once again, we have 



^A(Im,[/^0. (49) 
fc=i 

Let 9(AVi,t/fc) = DV(^fc-i)(^(^fc-i)l7fc)® 3 . Since sup fc E\e(X k _ u U k )\ 2 < oo and 
0, we have 

m Y, X T[ Q { x k-i,U k )-E{@{X k . l ,U k )\T k -x)]^Q. (50) 



lining Al 3) /(Al 3/2) ) 2 = 0, we have 



fc=i 



,(3/2) 

Observe that E(0(Xfe_i, £/fc)|.Fk_i) = J(Afe_i), where J is given by 

J(s) := / D 3 (l)(x)(cj(x)u)® 3 fi{du) 



Since A^ 2 -* -> oo as n -> oo, we can apply Theorem I1.3I to the measure v n = A ( 3 1 / 2 ) Sfe=i ^fc^ 2< ^ fc _i- 

Since J is continuous and bounded, we have lirm^oo v n { J) = J Jdv a.s., and the lemma follows on 
combining this fact with (|34" ]) -(|50 ]) . □ 



We are now ready to prove Theorem 1 1.5 1 

Proof of Theorem 11.51 The proof is similar as the proof of Theorem 9 of [25], once again the main 
difference is in the treatment of reflection terms. Using the notation of Lemma 13.11 we first observe 
that, for any sequence of positive numbers {a„}„ e N such that lirrin-^oo a„ = oo, we have Zn /a n — > 
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in probability. This is because, from Lemma \2 .41 the sequence {X n } n ^ is tight, and consequently so is 
{cj)(X n )} nf z K as well. 

We also derive from the definitions of Z n x \ Z n 2 ^ and Z n 3 ^ the inequalities 

E|4 1 )|< Kl ^A||| J DVlU, (51) 

and 

n 

E|ZW| 2 < K1 ^A^||^||L, i = 2,3, (52) 

k=l 

for some k\ £ (0, oo), for all n > 1. 

(a) Now assume that lim n _ i . 00 (l/ v / A^)Al 3 ^ 2 ' ) = 0. We then have Ymin^^ 5Zfe=i ^k/V^n = 0, an d it 
follows from (EJ) that Z^/y/A^ 0. We also deduce from ([52]), that Z^P/y/A^ 0, for j = 2,3. 
Consider now zffi . Denoting the Lipschitz norm of D 2 <p by L, we have 

\M*k-i,Xk)\ < ||A*fc| 3 < ^a 3 K 3 (X k + \/\~k~\U k \) 3 , 

where the second inequality follows from the Lipschitz property of the Skorokhod map (Condition II. ip . 
Thus, there exists k-i € (0, oo) such that, for all n > 1, 



E|Z( 4 )|< K2 ^A 3 / 2 , (53) 
fc=i 



and therefore Z n /y/Xn — > 0. 

We now, consider Y„ , for j = 1, 2, 3, 4. 



= ]T(V0(AVi),y fc -i) = £ A#X*-i)4-i- 
fc=i fe=i 

From we have < ^3 %/A/7( I t^fc I + !)• Also, for any fixed i, L l k _ 1 is not equal to only if there 

exists x <E Fi, such that HA^-i — x\\ < a\K\k + aiKy/~\k\Uk\] and in that case, using Taylor's theorem 
and the Lipschitz property of D 2 cf>, there exists K4 £ (0,oo), such that, 

\Ditf>(X k - X ) - Di4>(x) - (X fc _ x - x) T D 2 cf>{x)d i \ < K 4 ||Jf fc -i - x\\ 2 . 

Combining this with (fTU)) . we have 

|A^fc-i)| < «4||^fc-i ~x\\ 2 . 

Thus we have 

n 

E|yW|<« 5 ^Af , (54) 
fc=i 

for some constant K5. Using similar arguments as above, we obtain: 

n 

E|F„ (i) | < K5^2 ^fe /2 ' J = 2,3,4. (55) 
fe=i 

Thus we have that Y^/y/K^ ±U 0, for j = 1, 2, 3, 4. 

From Lemma |2~51 and recalling the definition of V (see (|T5)) ) we have that, for every £ e (0,co7), 

sup / e c ' a: 'i/„((ia;) < 00, a.s. 

raGN J G 

For such a C, under the assumption that limui^oo e - ^' \ Vcf>(x)\ 2 = 0, applying Lemma l3.21 we now have 

" Af( 0, / \a T V(j)\ 2 dis 



G 
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This completes the proof of part (a). 

z^^ - ('3/2') ~ 

(b) Assume now that ]im n ^ 00 (l/y/An)A.n = A G (0, +00]. We then have that 

n 

lim A<f /2) = +00 and lim VA^/A<f /2) = 0. 

n — yrm n — inn ^ — J 



n— yoo ' 

k=l 



As before, Z^ ] /A ( n /2) ^ 0. It follows from fl3T) that z£ ] /A% /2) 0, and from that 

Z®/Ag /2) Ao,for j = 2,3. 

Under the assumptions of part (b) (i.e. that D 3 is bounded, Lipschitz and (fTTj) holds), we have, using 
similar arguments as in part (a), for some € (0,oo), 

n 

E|r„ (j) | < K6 ^A 2 , j = l,...,4; n>l. (56) 
fe=i 

Thus we have that /A ( n /2) 0, for j = 1, 2, 3, 4. 

Applying Lemma I3~2l once again, we have, for <p satisfying limi a .|_ >00 e - ^' | V0(x)| 2 = 0, 

Nn C >N (o, [ \a T V^\ 2 dv\ . (57) 



\ JG 

Also from Lemma [3~3l 

^Tm ^ I I I Dmx){a{x)ur^{du)u{dx) = -m. (58) 

Now, if A < +oo, we have from the above observations that Zn /V^n ~> 0, for j = 0,1,2,3, 



Y# ] /VK ^> 0, for j = 1, 2, 3, 4 and 



7 (4) 



^ -Am. (59) 



The statement in ([12"]) now follows on combining this with ([5 

Finally, if A = +oo, we have Z n n /A ( n /2) ^ 0, for j = 0, 1, 2, 3, Y,[ f) /A ( n /2) ^ 0, for j = 1, 2, 3, 4 and 
N n /A n 3/2} ^ 0, and ([131) follows from ([58]). This completes the proof of Theorem □ 



4. Numerical Results 



4.1 Evaluation of the Euler Time Step. A key step in simulating the sequence {Xk} in is 
the evaluation of S(Xj~, Yfc+i — Xk), where S : G x R m — > G is the time-1 Skorokhod map defined in ([7]). 
In this section we describe a procedure for computing S(x, v) that uses well known relationships between 
Skorokhod problems and linear complementary problems (LCP). We restrict ourselves to a setting where 
N = m and G = R™. We begin by recalling the basic formulation of the LCP (see [ID])- For j G N, a 
j x j matrix R and a j-dimensional vector the LCP for (R, 9) is to find vectors u, v £ W such that 

u > 0,v > 0; 
7; = 6> + 
u ■ v = 0. 

It is well known (see [TB] and [5]) that with R = [di, d m }, under Condition 11.11 for every 6 £ R m , 
the LCP for (R, 9) admits a unique solution (u,v) = (£^(i?, 9), C 2 n {R, 9)), and furthermore C 2 n (R 7 9) = 
5(0, 9). Thus the evaluation of 5(0, 9) reduces to solving the above LCP for which numerous algorithms 
are available. In the examples considered in the current work we used a quadratic programming algorithm. 
Evaluation of S(x, 9) for x ^ can be carried out using a localization procedure as follows. 

Fix x e G and let J = ln(x) = {j G {1, m}\(x, ej) = 0}. Let Pj = {z e R m |(z, ej ) = 0, Vj e J c }. 
Let 7Tj : R m — > Pj be the orthogonal projection: 

7Tj(z) = 21 - ^ ( Z ^j) e j- 
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Suppose that |J| = p and J = {ii,...,i p }. Define a p x p matrix Rj be the relation Rj(k,l) = 
{'Kjdi l )i k , for k, I = l,...p. Let uj,vj £ K p be the solution of LCP for (Rj,ttj9), i.e., (uj,vj) = 
(Cj J (Rj,nj9) 7 Cp(Rj,TTj9)). Once again unique solvability of LCP for (Rj,ttj9) is assured from Condi- 
tion ll.ll Denote uj = (rji, ...,%,) and define xi(t) = x + 9t + tX^=i Vjdij- Let 

n = inf{i > 0\In(xi(i)) ^ In(x)}. 

We define t\ = oo if the above set is empty. Then T(x + 9i)(t) = x\(t) for all t < t\. If T\ < oo set 
the initial point to be x\ = x\(t\) and define the trajectory {x2{t)}t>o m a similar way as {x±(t)} by 
replacing x with x\. Set r 2 = inf{i > 0|In(x2(t)) 7^ In(:ci)}. Then 

T(x + 6i)(ri + t) = V(x 1 + 9i)(t) = x 2 (t), for all t < r 2 . 

Define now recursively trajectory {xj(t)} with time points Tj and end points Xj(rj), j = 3,4, .... Let jo 
be such that J^lLi n<l< EliV n- Then 

jo 

S(x,0) = T(x + 9i)(l) = T(x JO +9i)(l-J2^)- 

i=l 

Thus the evaluation of S(x, 9) can be carried out by recursively solving a sequence of LCP problems. 

One difficulty in implementing the above scheme is the possibility that Y^iLi T i — 1- However using 
regularity property of the Skorokhod map, we see that this occurs only when S(x, 9) is zero. Thus in the 
practical implementation of the algorithm we fix a finite threshold L and carry out the above recursive 
procedure at most L times and set S(x, 9) = if ti < 1. 



4.2 Results. 



4.2.1 A 3-d Example with Product Form Stationary Distribution. Let m = 3 and suppose 
that the reflection matrix is of the form R = I + Q, where / is the identity matrix, and Q is given as 



Q = 



0.1 -0.2 
-0.1 
0.2 



It can be checked that the spectral radius of Q is less than 1, and so Conditions 11.11 and 11.21 hold. Take 
the drift function b(x) = [— 1/2, — 1/2, — 1/2] T and a(x) = I, x £ K?_. The stationary distribution v 
for this example is of product form (see [H]): exp(1.1667) g) exp(1.0938) ® exp(0.8537), where exp(^) 
is the exponential distribution with parameter /1. In implementing the above numerical scheme, we set 
our initial point to Xq = [1, 1, 1] T , and simulate {Xk}^ =1 defined by equation ©, taking Ut ~ Af(0,I), 
Afe = \j\f~k and n = 10 7 . Figure Q] shows the comparison between the exact distribution with the 
first-coordinate marginal of the measure u n . 



4.2.2 Effect of Choice of {Afe}. Consider a two-dimensional SRBM with covariance matrix <j(x) 
I, drift vector b(x) = [— 1,0] T and reflection matrix 



R 



1 

-1 1 



This example was considered in |llj . We consider the first moment of the xi-coordinatc. The exact value 
for this moment is known to be 0.5. We consider Afe = k~ a and examine the influence of the choice of 
a on the numerical performance. The results are given in Figure [2j We find that a = 0.5 gives the best 
numerical convergence. 

4.2.3 An 8-d symmetric SRBM. A SRBM is said to be symmetric if its covariance matrix L, drift 
vector [i and reflection matrix R are symmetric in the following sense: Tij = Tji = p for 1 < i < j ' < d, 
fj,i = — 1 for 1 < i < d and Rij = Rji = —r for 1 < % < j < d, where r > . The positiveness of T implies 
— l/(d — 1) < p < 1 and the completely-5 condition of R implies r{d — 1) < 1. In this case, It is known 
(see [TT]) that, the first moment of each of the component is the same, and is given by the following 
formula 

1 - (d-2)r+(d-l)rp 

mi = 2(TT7) ■ 
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Figure 1: Comparison between the nnumerically computed distribution with exact distribution. The left 
figure shows the comparison between the empirical cumulative distribution function (cdf) and the exact 
cdf. The middle figure makes a comparison between the estimated density function and the exact density 
function. And the right figure is the qq-plot of the empirical quantiles versus the exact quantiles. 



1.4 - 

1.2 - 



0.8 - 




Figure 2: We consider time step sequence A„ = nT a with different choice of a and study the influence of 
a on numerical convergence. The thin solid line, the dotted line, the thick solid line, the dash-dot line, 
and the dashed line correspond to a =0.1, 0.3, 0.5, 0.7, and 0.9 respectively. The x-axis shows the value 
of n while the y-axis corresponds to J XiV n (dx). 



Here we take d = 8. Then the conditions on the data yield — 1/7 < p < 1 and < r < 1/7. Letting p 
range through {—0.1, —0.05, 0, 0.2, 0.9}, and r take value 0.1, we obtain estimates of mi using algorithm 
in this work. We take = fc~ a , a = 0.5 and n = 10 7 . The results are shown in Table [TJ The results 
show that as the correlation coefficient p approaches 1, the performance of the algorithm deteriorates. 
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Table 1: Estimates for mi when d = 8. 



p 


-0.1 


-0.05 





0.2 


0.9 


Estimated Val. 
True Val. 


0.131 
0.150 


0.137 
0.166 


0.163 
0.182 


0.414 
0.246 


3.205 
0.468 



Appendix A. Appendix 

Lemma A.l Let U be a random variable with bounded support. Suppose that EC/ = 0. Then there exists 
a € (0,oo), such that 

Proof. Without lots of generality we assume that \U\ < 1. 
Using the convexity of the function e Xx , we have 

\u U+l x 1-U \ 

e xu < e A + e" A . 

~ 2 2 

Taking expectations in the above inequality and using Taylor's expansion, we have 

^ \u e A + e _A a 2 
2 

The lemma then follows on taking a = i. □ 
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